Wide dynamic range operation for CMOS sensor with freeze-frame shutter

ABSTRACT

Wide dynamic range operation is used to write a signal in a freeze-frame pixel into the memory twice, first after short integration and then after long integration. The wide dynamic range operation allows the intra-scene dynamic range of images to be extended by combining the image taken with a short exposure time with the image taken with a long exposure time. A freeze-frame pixel is based on voltage sharing between the photodetector PD and the analog memory. Thus, with wide dynamic range operation, the resulting voltage in the memory may be a linear superposition of the two signals representing a bright and a dark image after two operations of sampling.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.10/053,110, filed Oct. 26, 2001 now U.S. Pat. No. 7,050,094, which inturn claims priority under 35 U.S.C. 119/120 from provisionalapplication Ser. No. 60/243,898 filed Oct. 26, 2000, both of which areincorporated in its entirety by references here.

TECHNICAL FIELD

This invention relates to CMOS sensors, and more particularly tooperating with a wide dynamic range using freeze-frame shutters.

BACKGROUND

An active pixel sensor (“APS”) is a special kind of light sensingdevice. Each active pixel includes a light sensing element and one ormore active transistors within the pixel itself. The active transistorsamplify and buffer the signals generated by the light sensing elementsin the pixels. One type of such APS devices is disclosed in U.S. Pat.No. 5,471,515 by Fossum et al., the disclosure of which is incorporatedherein by reference.

There are many applications for active pixel image sensors, includingscientific, as well as commercial and consumer applications. The specialtechniques of active pixel sensing allow using a semiconductor familyformation process which is compatible with CMOS, e.g., NMOS. Thistechnique enables the readout electronic to be integrated on the waferusing a similar process. The result is a high performance sensor withhigh quantum efficiency and low dark current.

CMOS active pixel image sensors may be operated using a “rolling”shutter. Such a shutter operates by reading out each row of pixels, andthen resetting that individual row, and then rolling to read and thenreset the next row of pixels. Each pixel hence gets read and then resetat slightly different times. Hence, each pixel has a slightly differenttime of integration. Some applications, such as high-speed photography,may require more time consistency than is possible using this approach.Therefore, in these other applications, a frame shutter may be used. Inthe frame shutter mode, all pixels in the array have substantiallyidentical integration start times and integration stop times.

A wide dynamic range (WiDyR) technique was developed for CMOS sensorswith a rolling shutter and is described in U.S. Pat. No. 6,115,065,which is incorporated herein by reference. The WiDyR technique allowsthe extension of the intra-scene dynamic range of images by combining animage taken with a short exposure time with an image taken with a longexposure time. U.S. Pat. No. 6,115,065 teaches designs and operationalmethods to increase the dynamic range of image sensors and APS devicesin particular by achieving more than one integration times for eachpixel thereof. An APS system with more than one column-parallel signalchains for readout are described for maintaining a high frame rate inreadout. Each active pixel is sampled for multiple times during a singleframe readout, thus resulting in multiple integration times. Theoperation methods can also be used to obtain multiple-integration timesfor each pixel with an APS design having a single column-parallel signalchain for readout. Furthermore, analog-to-digital conversion of highspeed and high resolution can be implemented.

SUMMARY

Wide dynamic range operation is used to write a signal in a freeze-framepixel into the memory twice, first after short integration and thenafter long integration. The wide dynamic range operation allows theintra-scene dynamic range of images to be extended by combining theimage taken with a short exposure time with the image taken with a longexposure time. A freeze-frame pixel is based on voltage sharing betweenthe photodetector PD and the analog memory. Thus, with wide dynamicrange operation, the resulting voltage in the memory may be a linearsuperposition of the two signals representing a bright and a dark imageafter two operations of sampling.

DESCRIPTION OF DRAWINGS

These and other features and advantages of the invention will becomemore apparent upon reading the following detailed description and uponreference to the accompanying drawings.

FIG. 1 illustrates one example of a freeze-frame pixel that may be usedto obtain samples according to the present invention.

FIG. 2 illustrates sampling without using the wide dynamic range processaccording to one embodiment of the present invention.

FIG. 3 illustrates sampling using the wide dynamic range processaccording to one embodiment of the present invention.

FIG. 4 illustrates an idealized transfer curve which is achievable usingthe present invention.

DETAILED DESCRIPTION

A freeze-frame pixel 100 is shown in FIG. 1 and comprises aphotodetector PD with capacitance C1, an analog memory C2, a sourcefollower transistor SF, switches S1–S3, and a row select switch RowSel.The photodetector PD with capacitance C1 is connected to a reset voltageV_(rst) through the switch S1. The analog memory C2 is connected to thereset voltage V_(rst) through switch S3. The photodetector PD withcapacitance C1 is connected to the analog memory C2 through the switchS2. The pixel structure 100 allows simultaneous photo-detection andreadout of data stored in the pixel memory during the previous frame.

The typical sequence of operations (without WiDyR) functions 200 isillustrated graphically in FIG. 2. The switches S1 and S2 are enabled byactive low pulses 205, 210, respectively. The analog memory C2 is resetduring a previous readout time through switch S3. To start an exposure215, the photodetector PD is reset through S1 at the first low pulse inpulse train 205. To complete the integration 220 and transfer the datato the memory, the switch S2 is closed at the first low pulse in pulsetrain 210, thereby connecting the photodetector PD to the analog memoryC2. The readout of the data from the pixel is done row by row. After thepixel signal is read 225, the analog memory is reset through S3. Thereset level is also read out to subtract the pixel source followeroffset voltage.

The sequence of operations with the wide dynamic range process 300 isillustrated graphically in FIG. 3. The switches Si and S2 are enabled byactive low pulses 305, 310, respectively. In wide dynamic rangeoperation, the idea is to write the signal into the memory twice, firstafter short integration and then after long integration. The pixel 100in FIG. 1 is based on voltage sharing between the photodetector PD andthe analog memory C2. Thus, after two operations of sampling, theresulted voltage in the memory will be a linear superposition of the twosignals representing bright and dark image.

The wide dynamic range operation process 300 is performed by resettingthe analog memory capacitor C2 to V_(rst) through the switch S3. Theanalog memory capacitor C2 may also be reset during the previousreadout. The process 300 then resets the photodetector PD to V_(rst)through the switch S1. The photodetector PD then begins integration of asignal to create a photocharge Q1 at time 315. After a short integrationperiod t1, the signal is sampled to the analog memory capacitor C2 byenabling the switch S2 for a short time. The photodetector PD continuesto integrate the signal for a long integration period t2 to create aphotocharge Q2. Following the long integration period t2, the signal isagain sampled to the analog memory capacitor C2 by enabling the switchS2. The signal is then read out 330 to all memories in the entire pixelarray.

Because of the capacitor divider effect, after two samplings from thephotodetector PD, the resulted signal voltage will be a combination ofthe signal accumulated during the short integration period t1 and thelong integration period t2. For corresponding photocharges Ql and Q2 atthe photodetector PD, respectively, the signal voltage will be equal to:V _(signal) =Q1*C2/(C1+C2)² +Q2/(C1+C2).

The signal voltage V_(signal) is a weighed sum of the photocharges Q1and Q2, so that the short integration is given a weight lesser by amountC2/(C1+C2).

The option of having wide dynamic range is useful when a portion of theimage after the long exposure is saturated. When the long exposure issaturated, the resulting response after summing the two signals will bethe following:V _(signal) =Q1*C2/(C1+C2)² +Q _(sat)/(C1+C2);

where Q_(sat) is the saturation charge for the photodetector PD.

Because the memories hold the short integration signal, the overlap ofthe long integration and the frame readout are no longer possible.However, this is not a problem for a pixel with a synchronous shutter.Typically the shutter is needed for making very short snaps, at least 10times shorter than the frame time. Then the relative increase of frametime including the exposure will not be substantially lower than theframe rate of the image sensor.

The graph 400 in FIG. 4 illustrates the idealized transfer curve whichis achievable using the present invention. The resulting slope for theshort integration is different from the original short exposure slopebecause of the C2/(C1+C2) factor.

However, one of the problems with the current pixel is that thesaturation voltage for photodiodes, approximately 0V, is different fromthe pixel saturation voltage (−0.7 V for nMOS source follower SFtransistor). Also, the analog memory capacitance C2 is onlyapproximately ⅕ of the total capacitance of C1+C2. Therefore, the kneeof the combined transfer curve is below the threshold of the sourcefollower transistor SF and we do not see an improvement. Another problemcan arise from the bulk charge. If there is a noticeable charge left inthe pixel after saturation, then it can saturate the memory during thesecond sample and thus erase the signal stored in the analog memory C2after short integration.

These problems can be addressed to achieve the dual-slope wide dynamicrange response. First, the source follower SF and rowselect RowSelMOSFETS can be changed to p-MOS type transistors. Second, if thesignal-to-noise ration is not a problem, the size of the memorycapacitor C2 may be increased so that the knee of the combined transfercurve is above the threshold of the source follower transistor SF.Finally, using an antiblooming gate for the photodetector PD withpotential artifact such as fixed pattern noise due to antibloomingthreshold variation.

The present invention may be used with other pixels that do not use thefreeze-frame structures, such as a photogate, with charge transferrather that voltage sharing. The capacity of the detector at saturationshould not then exceed the capacity of the memory to avoid erasing ofthe signal kept in the analog memory C2 after short integration.

Numerous variations and modifications of the invention will becomereadily apparent to those skilled in the art. Accordingly, the inventionmay be embodied in other specific forms without departing from itsspirit or essential characteristics.

1. A method of obtaining an image signal using a solid state sensor, themethod comprising: collecting a short image signal during a first timeperiod; sampling the short image signal after the first time period;collecting a long image signal during a second time period; sampling thelong image signal after the second time period; and combining the shortimage signal and the long image signal in a memory in the sensor tocreate a total image signal.
 2. The method of claim 1, wherein thesecond time period includes the first time period.
 3. The method ofclaim 1, further comprising resetting a photodetector in the sensorprior to collecting the short image signal.
 4. The method of claim 1,further comprising resetting the memory containing the total imagesignal prior to collecting the short image signal.
 5. The method ofclaim 1, further comprising simultaneous sampling of the short imagesignal while collecting the long image signal.
 6. The method of claim 5,wherein the short image signals and the long image signals are notcollected during the reading of the total image signal.
 7. The method ofclaim 1, wherein the memory is a capacitor.
 8. The method of claim 7,wherein the acts of collecting include operating a switch thatselectively connects said capacitor to a photodetector.
 9. The method ofclaim 1, wherein the second time period is longer than the first timeperiod.
 10. A pixel sensor comprising: a photodetector having a firstmemory element; a second memory element; and a plurality of switchesincluding: a first switch constructed to connect the photodetector to areset voltage source; a second switch constructed to connect thephotodetector to second memory element for permitting transfer of afirst and a second image signal collected in the photodetector during arespective first and second collection time period; and a third switchfor connecting the second memory element to a reset voltage source,wherein the third switch is different than either the first or thesecond switch, and wherein the second memory element is constructed suchthat it is able to combine the first image signal and the second imagesignal to create a total image signal.
 11. The pixel of claim 10,wherein the second time period includes the first time period.
 12. Thepixel of claim 10, further comprising a readout circuit to readout thetotal image signal from said second memory element.
 13. The pixel ofclaim 10, wherein each of said first memory elements and said secondmemory elements comprise capacitors.
 14. A method of operating an imagesensor comprising: resetting a capacitive structure by activating afirst switch by connecting the capacitor to a reset voltage source;resetting a photodetector of the image sensor by activating a secondswitch; integrating charge at the photodetector during a firstintegration period to generate a first image signal; transferring thefirst image signal from the photodetector to the capacitive structure byactivating a third switch; integrating charge at the photodetectorduring a second integration period to generate a second image signal;transferring the second image signal from the photodetector to thecapacitive structure by activating the third switch; and creating atotal image signal by combining the first and the second image signalsin the capacitive structure.
 15. The method of claim 14, furthercomprising reading out the total image signal through a readout circuit.